Rate control is an integral component of modern video compression systems. A compressed bit stream satisfies specific bandwidth and delay constraints that are imposed by the transmission medium. These constraints can be satisfied through rate control. Rate control algorithms vary the number of bits allocated to each picture, color component, set of blocks, or individual blocks, so that encoder and decoder buffer overflows and underflows may be avoided, and the target bit rate may be achieved with high accuracy. Optionally, a rate control algorithm may attempt to maintain consistently high visual quality. Such above-mentioned goals may compete with each other, which may contribute to a complex optimization problem. Notwithstanding, pictures in a compressed video bit stream can be encoded in a variety of arrangements, using for example, intra-predicted (I), uni-predicted (P), and bi-predicted (B) slices. A slice comprises one or more macroblocks in a picture that are coded using the same prediction type: I-, P-, or B-. A picture can be coded using a single or multiple slices, which do not have to be of the same type. For rate control purposes, a slice may be further subdivided into one or more rate control basic units (e.g., a collection of macroblocks), which are coded using the same coding parameters. A basic unit may be defined using a raster scan order, may be arbitrary, or may depend on objects within a scene. A basic unit may also be a collection of adjacent pictures. It should be appreciated that within this document the terms basic unit, slice, picture, and frame may be used interchangeably, depending on context. Bit rate and quality statistics may vary greatly between different types of slices. Modem rate control algorithms may gather statistics for rate control basic units of a single slice type and apply the gathered statistics on rate control basic units of such type alone, while for other slice types heuristic rate control methods may be employed. Performing rate control on slice types or basic units may employ separate and possibly different rate control models, which may achieve more accurate rate control and potentially improve the visual quality of the compressed bit stream. However, using separate rate control models may impose a computational cost.
An objective of a video compression system is high compression performance, e.g., to achieve the lowest possible distortion given a fixed target number of bits for the compressed bit stream or equivalently to send the minimum possible number of bits for a fixed distortion value. An example of distortion measurement is peak signal-to-noise ratio (PSNR). Video encoders produce a compressed bit stream such that when a compliant decoder mechanism (decoder) decodes the compressed bit stream, a reconstructed video sequence is generated that can be displayed and viewed at a receiver side. It should be appreciated that a storage or transmission medium can send such compressed bit stream to a receiver to be decoded in a variety of transport modes. Each one of these transport modes can have different delay and bandwidth applications. For example:                (a) the compressed bit stream can be stored and transported on an optical or magnetic disk, or on non-volatile computer memory;        (b) a user can download the compressed bit stream from some remote server on the internet and view the reconstructed video off-line at some later time;        (c) the user can stream the compressed bit stream from a remote internet server and view the compressed bit stream with a small time delay on the client computer or device; and        (d) the compressed bit stream can be the result of either real-time interactive video communication (e.g., video-conferencing) or live-event video streaming (e.g., sports), etc.        
For real-time communication, low end-to-end delays may improve quality of service to users. Somewhat higher end-to-end delays may be expected for live-event streaming, videoconferencing, etc. Optical and magnetic disk storage and movie downloads may tolerate greater delays, e.g., because decoding and display on a computer may have available buffering space. It has been found that internet streaming of movies or TV shows can allow for additional delay compared to live-event streaming and render a same amount of satisfaction to a viewer. It has also been found that end-to-end delay may be affected by the communication channel (or “pipe”) as well as video coder prediction structure. For instance, some video coders buffer future frames prior to coding the current frame to improve compression performance. However, buffering future frames may increase delay.
It should be appreciated that the capacity of the data pipe can vary for each transport medium. For example, optical and magnetic disk storage can be very generous in terms of bandwidth. High-capacity storage mediums such as Blu-Ray or HD-DVD disks have an upper limit on bit capacity and decoder buffer size. Off-line playback may not be constrained with respect to bandwidth, because the bit stream is viewed offline. However, practical configurations may relate to buffering delay and hard drive storage space. Internet streaming and real-time interactive video communication can be affected by the bandwidth of the networks used to transport the bit streams. In addition, bit streams generated for one transport medium might not be suitable for transmission through a different transport medium. For example, a bit stream that is stored on an optical disk (e.g., DVD) may be compressed at a bit rate such as 5 Mbps. Streaming the same bitstream online over a network that lacks bandwidth to support such a bit rate may affect performance.
A mechanism for generating compressed bit streams that satisfy the bandwidth and delay constraints is rate control. Rate control may minimize visual distortion, generates bits that fit the communication channel, and may deter decoder input buffer overflow or starvation (e.g., that may result in an underflow.) For instance, a compliant bit stream may satisfy two constraints with respect to the decoder buffer:                (a) received picture bits fit in the buffer (otherwise results in a buffer overflow), and        (b) when the decoder removes a picture from the buffer to decode the picture, the picture is received in its entirety (otherwise results in buffer underflow or starvation.)        
The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section.